# Low-Resource Inference

Motif 2.6B is a language model with 2.6 billion parameters, trained from scratch on AMD Instinct™ MI250 GPUs, aiming to build AI that aligns with human values, is useful, and reliable.

This is a direct GGUF conversion version of the bytedance-research/Phantom model, usable in ComfyUI with the ComfyUI-GGUF custom node.

This model is a GGUF format conversion of llava-hf/llava-1.5-7b-hf, supporting image-to-text generation tasks.

This is a GGUF format model converted from ByteDance-Seed/Seed-Coder-8B-Reasoning-bf16, suitable for code generation and reasoning tasks.

Qwen3 is the latest iteration of the Tongyi Qianwen series of large language models, offering a complete suite of dense models and Mixture of Experts (MoE) models. Based on large-scale training, Qwen3 achieves breakthrough progress in reasoning capabilities, instruction following, agent functionalities, and multilingual support.

The Linglong Linear Preview is a hybrid linear sparse large language model open-sourced by InclusionAI, with a total of 17.1B parameters and 3.0B activated parameters. This model implements long-text reasoning based on a hybrid linear attention mechanism, achieving near-linear computational complexity and near-constant space complexity during inference.

A fine-tuned text-to-speech model based on Canopy Labs' pre-trained model, quantized for efficient inference

A text-to-speech (TTS) model fine-tuned from Canopy Labs' pre-trained model, quantized in GGUF FP16 format for efficient inference

Phi-4-mini is a quantized language model developed by the PyTorch team, featuring 8-bit embeddings, 8-bit dynamic activations, and a 4-bit weight linear layer (8da4w) quantization scheme, making it suitable for mobile deployment.

This model is a GGUF-format conversion of Qwen2.5-VL-7B-Instruct, supporting multimodal tasks and applicable to image and text interaction processing.

This is the GGUF quantized version of the Qwen2.5-VL-7B-Instruct model, suitable for multimodal tasks and supports both image and text inputs.

A large language model based on the Phi 4 architecture, with 14 billion parameters, optimized for natural language processing and text dialogue tasks.

This is a quantized version based on the mlabonne/gemma-3-4b-it-abliterated model, using llama.cpp for imatrix quantization, suitable for image-text-to-text tasks.

RWKV-7 model using Flash Linear Attention format, trained on the Pile dataset, supporting English text generation tasks.

Quantized version of OlympicCoder-32B, based on llama.cpp's imatrix quantization method, suitable for code generation tasks.

The GemmaX2-28-2B GGUF quantized model is a series of quantized variants based on GemmaX2-28-2B-v0.1, designed for multilingual machine translation and supports 28 languages.

Quantized version based on ozone-ai/0x-lite model, supporting Chinese and English text generation tasks, using llama.cpp for imatrix quantization, offering multiple quantization options to adapt to different hardware requirements.

This is an 8B-parameter model based on the Llama-3.1 architecture, specifically optimized for Uzbek and English, supporting tasks such as text generation, summarization, translation, and Q&A.

The RWKV-7 model adopts a flash linear attention architecture and supports multilingual text generation tasks.

This is a GGUF format model converted from the Qwen2-VL-7B-Captioner-Relaxed model, specifically designed for image-to-text tasks.

SenecaLLM is a large language model fine-tuned based on Qwen2.5-Coder-7B-Instruct, specializing in cybersecurity-related tasks.

Llama 3.1 8B Instruct is a large language model released by Meta. It has been fixed and optimized for a specific version, improving performance and compatibility.

This is a version of the Llama-3-8b model quantized to INT4, using the QQQ quantization technique with a group size of 128 and optimized for hardware.

This model is a GGUF format conversion based on xtuner/llava-llama-3-8b-v1_1, supporting multimodal interaction between images and text.

Eris Prime V2 is a 7B-parameter multimodal language model with vision capabilities, requiring Koboldcpp for operation.

GGUF quantized version of Mixtral-8x7B-v0.1, supporting multiple bit quantization, suitable for text generation tasks.

The GPTQ quantized version of Deepseek Coder 1.3B Instruct, offering multiple quantization parameter options, suitable for code generation and computer science-related tasks.

HHEM-2.1-Open is a hallucination detection model developed by Vectara, designed to evaluate the consistency between content generated by large language models and given evidence.

Llava v1.5 13B is a multimodal model developed by Haotian Liu, combining visual and language capabilities to understand and generate content based on images and text.

The Mistral Trismegistus 7B Model is a professional-grade large language model specializing in esotericism, metaphysics, and spirituality. It is based on the Mistral-7B architecture and fine-tuned with synthetic data generated by GPT-4.

CodeLlama 7B is a 7B-parameter code generation and comprehension model developed by Meta, optimized based on the Llama 2 architecture and focused on programming tasks.

MythoMax L2 13B is a large language model developed by Gryphe, based on the Llama 2 architecture, focusing on role-playing and creative text generation.

A 2.7 billion parameter code generation model developed by Replit, supporting 20 programming languages

Cadet-Tiny is an ultra-compact dialogue model trained on the SODA dataset, specifically designed for edge device inference, with a size of only about 2% of the Cosmo-3B model.

A QA model fine-tuned on SQuAD v1 based on BERT-base uncased, with 66% of linear layer weights pruned via nn_pruning library, achieving 2.32x inference speedup